-
Notifications
You must be signed in to change notification settings - Fork 161
Docs: flowey: Add developer/contributor guide for working with flowey #2278
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds a comprehensive developer guide for working with flowey, the OpenVMM project's automation framework. The guide introduces core concepts, architectural patterns, and best practices to help contributors understand and work with flowey effectively.
Key changes:
- Added detailed documentation explaining flowey's two-phase execution model and backend abstraction
- Documented core concepts including pipelines, artifacts, nodes, and variables
- Included practical examples and design patterns for common workflows
mattkur
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is already useful, thanks! I learned a few things reading through this.
Overall, my commentary is primarily stylistic, since the content either already matches what I know, or is telling me things I don't know.
My general style feedback is:
- Keep the
Guideconceptual, and move code and code examples into the rust docs. We publish those as part of the build, so you can even link to them directly from this guide. - Use ```admonish rather than the existing Note (for here and other important places)
Besides the semantic questions that I asked, I also wonder:
- Do we have code layout or naming conventions? (e.g.
_jobsfolder,artifact_*.rsfor artifact definitions) - How much processing should I do in an
artifact? Is it just a record of files? Or should I ever do a transformation in one?
daniel5151
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@smalis-msft sent this my way, and given that I didn't end up sending in flowey docs before I left msft (tbf, I had a very short runway prior to leaving!), I figured the least I could do is chime in with some comments here.
maybe the most important feedback is that you def want to invert the presentation of a lot of the concepts in the guide. start from the base building blocks (steps), go into how those compose into nodes (with requests), how those get coalesced into jobs, and how those become pipelines (with a discussion on artifacts). As written today - this guide is really hard to follow, unless you are willing to read this whole thing multiple times.
also, having a high-level overview discussing the "why" behind flowey seems important to set the stage for the rest of the discussion. i.e: talk about how traditional back + YAML based pipelines are impossible to maintain, rely heavily on non-local reasoning and global state, can't be run locally / ported between platforms, etc...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 11 out of 12 changed files in this pull request and generated 1 comment.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
|
||
| **[Pipelines](https://openvmm.dev/rustdoc/linux/flowey_core/pipeline/trait.IntoPipeline.html)** are the top-level construct that defines a complete automation workflow. A pipeline specifies what work needs to be done and how it should be organized. Pipelines can target different execution backends (local machine, Azure DevOps, GitHub Actions) and generate appropriate configuration for each. | ||
|
|
||
| **[Jobs](https://openvmm.dev/rustdoc/linux/flowey_core/pipeline/struct.PipelineJob.html)** represent units of work that run on a specific platform (Windows, Linux, macOS) and architecture (x86_64, Aarch64). Jobs can run in parallel when they don't depend on each other, or sequentially when one job's output is needed by another. Each job is isolated and runs in its own environment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Each job is isolated and runs in its own environment.
Not really true, yes they have some rust-level isolation, but they're still running on the same system and could rely on implicit global state that you forget to track. It's not like we're doing any really heavy sandboxing.
|
This looks fantastic, thanks for taking this on! Just a few little cleanups and I think it's good. |
Adding more content to our flowey documentation to help ease ramp up for contributors in this area